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(54) Compact high resolution panoramic viewing system 



(57) A compact high resolution omnidirectional or 
panoramic viewer has several cameras with a common 
virtual optical center. The field of view of each of the 
cameras is arranged to form a continuous 360 degree 
view of an area when taken as a whole. The cameras 
are positioned so that they each view a different reflec- 
tive surface of a polyhedron such as a pyramid. This re- 
sults in each camera having a virtual optical center po- 
sitioned within the pyramid. The cameras may be posi- 
tioned so that their virtual optical centers are offset from 
each other. The offsets produce narrow blind regions 



that remove image distortions received from the edges 
of the pyramid's reflective surfaces. The reflective pyr- 
amids may be stacked base to base or nested within 
each other to produce a more compact panoramic view- 
er. Using two or more reflective pyramids in such close 
proximity permits using many cameras with the same 
virtual optical center. Using many cameras divides a 
large viewing area into many smaller areas where an 
individual camera views each smaller area. Since each 
camera views a smaller area, increased resolution is 
provided to the user. 



FIG. 14 
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Description 

Cross Reference To Related Inventions 

[0001 ] This application is related to the following com- 
monly assigned US Patent Applications: "Panoramic 
Viewing Apparatus", Serial No. 08/431,356 which is- 
sued as U.S. Patent 5,745,305; "Method And System 
For Panoramic Viewing", filed April 28, 1995, Serial 
Number. 08/431354; "High Resolution Viewing Sys- 
tem", Serial No. 08/497673, which issued as U.S. Patent 
5,793,527; "Panoramic Projection Apparatus", Serial 
Number 08/497341, which issued as U.S. Patent 
5,539,483; "Spherical Viewing/Projection Apparatus", 
filed November 30, 1995, Serial Number 08/565501; 
"Panoramic Viewing System With Offset Virtual Optical 
Center", filed October 7, 1997, Serial Number 
08/946443; commonly assigned and concurrently filed 
US Patent Application entitled "Stereo Panoramic View- 
ing System"; and commonly assigned and concurrently 
filed US Patent Application entitled "Panoramic Viewing 
System With Support Stand". 

Background Of the Invention 

Field of the invention 

[0002] The present invention relates to a viewing sys- 
tem; more particularly, a panoramic viewing system. 

Description of the Related Art 

[0003] In an effort to operate more efficiently, it is de- 
sirable to perform some tasks using telepresence. Tel- 
epresence refers to providing visual or other types of 
sensory information from a device at a remote site to a 
user that makes the user feel as if he/she is present at 
the remote site. For example, many businesses now 
hold meetings using telepresence. Telepresence is also 
useful for distance learning and remote viewing of 
events such as concerts and sporting events. A more 
realistic telepresence is provided to a user by providing 
the user with the capability to switch between views, and 
thereby mimic, for example, looking around a meeting 
room. 

[0004] In the past, when several views were made 
available to a user, several cameras with different opti- 
cal centers were used. Such a situation is illustrated in 
FIG. 1. FIG. 1 illustrates cameras 2, 4, 6, and 8 with 
optical centers 10, 12, 14, and 16, respectively. When 
the user decided to change views, he or she simply 
switched between cameras. In more sophisticated sys- 
tems, when a user decided to change views, he or she 
was able to obtain a view from optical centers 10, 12, 
14, or 16 as well as from additional optical centers 18, 
20, 22, 24, or 26. Views associated with optical centers 
such as 18, 20, 22, 24, and 26 were obtained by using 
views from the two cameras nearest to the selected op- 
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tical center For example, a view from optical center 18 
was obtained by using the views from cameras 2 and 4 
and interpolating between the two views so as to simu- 
late a view from optical center 18. Such procedures in- 

s troduced irregularities into views. In addition, forming 
these interpolated views required a large amount of 
computational power and time, and thereby made this 
technique expensive and slow to respond to a user's 
commands. This computational overhead also limited 

io the number of users that could simultaneously use the 
system. 

Summary of the Invention 

is [0005] One embodiment of the present invention pro- 
vides an omnidirectional or panoramic viewer in which 
multiple cameras effectively have a common optical 
center at least one of these cameras having its field of 
view redirected by a planar mirror. The field of view of 
each of the cameras is arranged to form a continuous 
360 degree view of an area when taken as a whole. The 
user can sweep through 360 degrees of viewing, where 
each view has the same or nearly the same optical cent- 
er, by simply using the output of one camera, more than 
one or the combination of cameras without requiring the 
computational overhead of interpolation used in the pri- 
or art. Such an arrangement may be used to enhance 
use of virtual meeting rooms by allowing a viewer to see 
the meeting room in a more natural format. This format 
corresponds closely to a person sitting in the actual 
meeting who simply turns his or her head to change the 
view at a particular time. 

[0006] In another embodiment of the invention, the 
cameras are positioned so that they each view a differ- 
ent reflective surface of a solid or hollow polyhedron 
such as a solid or hollow pyramid. This results in each 
camera having a virtual optical center positioned within 
the pyramid. The cameras are positioned so that their 
virtual optical centers are offset from each other. The 
offsets produce narrow blind regions that remove image 
distortions received from the edges of the pyramid's re- 
flective surfaces. 

[0007] In still another embodiment of the invention, a 
stereo panoramic view is provided through the use of 
multiple virtual optical centers. A reflective polyhedral 
element, such as a pyramid, redirects the field of view 
of each camera in a first set of cameras to form a group 
of substantially co-located virtual optical centers at a 
first location within the pyramid. The pyramid also redi- 
rects the field of view of each camera in a second set of 
cameras to form a group of substantially co-located vir- 
tual optical centers at a second location within the pyr- 
amid. Panoramic images from the first and second vir- 
tual optical centers provide a stereo panoramic view 
when one panoramic image is provided to a user's left 
eye and the other panoramic image is provided to the 
user's right eye. 

[0008] In yet another embodiment of the present in- 



EP 0 982 946 A1 



25 



30 



35 



40 



45 



50 



3 



3 

vention, polyhedrons such as pyramids having reflective 
surfaces are stacked base to base or nested within each 
other to produce a compact panoramic viewer Using 
multiple reflective polyhedrons in such a manner per- 
mits using many cameras with the same or nearly the 
same virtual optical center. Using many cameras divides 
a large viewing area into many smaller areas where an 
individual camera views each smaller area. Since each 
camera views a smaller area, increased resolution is 
provided to the user. 

[0009] In another embodiment of the present inven- 
tion, the reflective polyhedron such as a pyramid is sup- 
ported by a post that passes through the vertex of the 
pyramid. Cameras are then mounted to the post to pro- 
vide a panoramic viewer with a mounting structure and 
a structure for supporting individual cameras. 
[0010] In still another embodiment of the present in- 
vention, a nearly spherical view is provided to a user by 
placing a camera at the common virtual optical center 
of the viewer. In order to enhance the spherical view, the 
camera at the common virtual optical center may use a 
wide angle lens. 

[0011] In yet another embodiment of the present in- 
vention, the viewing device may include any type of im- 
age processing device. If the image processing device 
is a camera or other type of image capture device, a 
panoramic image is captured for the user, and if the im- 
age processing device is a projector or other type of im- 
age producing device, a panoramic image is produced 
for the user. 

Brief Description of the Drawing 
[0012] 

FIG. 1 illustrates a prior art multiple camera viewing 
system; 

FIG. 2 illustrates a four camera omnidirectional or 
panoramic viewing system using a four-sided pyra- 
mid with reflective surfaces; 

FIG. 3 illustrates how a reflective surface of the pyr- 
amid is used to provide each camera with a com- 
mon optical center; 

FIG. 4 illustrates the top view of the pyramid illus- 
trating the camera positions; 
FIG. 5 illustrates an eight-sided pyramid with reflec- 
tive side surfaces; 

FIG. 6 is a top view of the pyramid of FIG. 5; 
FIG. 7 is a block diagram of a system to control data 
produced by the cameras; 

FIG. 8 illustrates the association between the data 
received from the cameras and the view presented 
to a user; 

FIG. 9 illustrates an addressing scheme for the 
memory of FIG. 7; 

FIG. 10 is a block diagram of the controller of FIG. 7; 
FIG. 1 1 illustrates the viewing system of FIG. 2 with 
a fifth camera; 
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FIG. 1 2 illustrates a top view of the pyramid of FIG. 
2 with displaced virtual optical centers; 
FIG. 13 illustrates the pyramid of FIG. 12 with 
shades positioned in blind regions; 
s FIG. 14 illustrates a panoramic viewer using pyra- 
mids stacked base to base; 

FIG. 15 illustrates a panoramic viewer using nested 
pyramids; 

FIG. 16 illustrates a spherical viewer using nested 
10 pyramids; 

FIG. 17 illustrates a stand used to support a pano- 
ramic viewer; 

FIG. 18 illustrate two types of distortion; 
FIG. 19 illustrates a calibration process; 
'5 FIG. 20 illustrates the association between data re- 
ceived from the cameras and the view presented to 
the user with distortion; 

FIG. 21 illustrates how distorted image data is 
stored; 

FIG. 22 illustrates how mapped image data is 
stored; and 

FIG. 23 is a block diagram of a panoramic camera 
system where image mapping is used. 

Description of the Preferred Embodiment 

[0013] FIG. 2 illustrates a four camera system for pro- 
viding a 360 degree view to a user, where the cameras 
each have a common or nearly common virtual optical 
center within the pyramid. Pyramid 40 has reflective 
sides 42, 44, 46, and 48 and may be a hollow, solid or 
truncated structure. In a preferred embodiment, each of 
the reflective sides forms a 45 degree angle with a plane 
parallel to base 50 and passing through the vertex of 
pyramid 40. Cameras 52, 54, 56, and 58 are associated 
with pyramid reflective surfaces 48, 42, 44, and 46, re- 
spectively. The cameras may be image gathering devic- 
es such as an optical scanner. As a result, camera 52 
views a reflection from surface 48 to enable it to view 
objects in the direction of arrow 60. Camera 54 views a 
reflection from surface 42 to view objects in the direction 
of arrow 62. Camera 56 views a reflection from surface 
44 to view objects in the direction of arrow 64, and cam- 
era 58 views a reflection from surface 46 to view objects 
in the direction of arrow 66. Each camera has a 90 de- 
gree field of view; however, larger fields of view may be 
used and the overlapping portion of the images may be 
removed by deleting or combining the pixels associated 
with the overlapping views. The combination of the four 
cameras viewing reflections from their associated re- 
flective surfaces on pyramid 40, produce a 360 degree 
view of the area surrounding pyramid 40. When the mir- 
rors are at 45 degrees with respect to the pyramid base, 
it is desirable to locate the optical center of each camera 
on a plane that is parallel to base 50 and intersects ver- 
tex 70 of pyramid 40. Each camera's optical center 
should also be located on a line that passes through ver- 
tex 70 and is perpendicular to the base line of the cam- 
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era's associated reflective surface. For example, the op- 
tical center of camera 54 is located on line 72. Line 72 
is perpendicular to base line 74 of reflective surface 42. 
Line 72 is in a plane that passes through vertex 70 and 
is parallel to base 50. Likewise, the optical center of 
camera 56 is positioned on line 76 which is perpendic- 
ular to baseline 78, the optical center of camera 58 is 
positioned on line 80 which is perpendicular to base line 
82, and the optical center of camera 52 is positioned on 
base line 84 which is perpendicular to base line 86. 
[0014] Each camera optical center is positioned on 
one of the above described lines at a distance X from 
vertex 70 and each camera has its optical axes or direc- 
tion of view pointing perpendicular to base 50. (The dis- 
tance X should be such that the reflective surface re- 
flects as much of the camera's field of view as desired; 
however, the defects in the reflective surface become 
more visible when the camera is moved closer to the 
reflective surface.) This positioning of optical centers re- 
sults in the cameras sharing a virtual optical center lo- 
cated at, or substantially at, position 90. Virtual optical 
center 90 is located a distance X from the vertex 70 on 
a line that passes through vertex 70 and is perpendicular 
to base 50. 

[0015] Although a pyramid configuration has been 
discussed in this example, different planar mirror ge- 
ometries may be used to redirect fields of view so that 
the cameras have virtual optical centers that are sub- 
stantially co-located. For example, solid, hollow or par- 
tial polyhedrons may be used. Additionally, in the case 
of a pyramid configuration the base and vertex do not 
have to be physically present and can be thought of as 
conceptual aids such as a base plane or end and vertex 
point or end. 

[0016] FIG. 3 illustrates another view of pyramid 40 
where only camera 54 is shown for the sake of simplicity. 
Camera 54 is positioned on line 72 so as to have a virtual 
optical center at, or nearly at, position 90 within pyramid 
40. If camera 54 has a 90 degree field of view in the 
direction perpendicular to base 50, and if the optical 
center of camera 54 is at a distance of X from vertex 70 
along line 72, camera 54 has a 90 degree view in the 
direction of arrow 62. In similar fashion, cameras 56, 58, 
and 52 have 90 degree views in the direction of arrows 
64, 66, and 60, respectively. This arrangement inexpen- 
sively produces a 360 degree field of view of an area 
because cameras with a 90 degree field of view have 
relatively inexpensive optics. 

[001 7] FIG. 4 is a top view of pyramid 40. FIG. 4 illus- 
trates the placement of the optical center of camera 54 
along line 72. Line 72 should be in a plane passing 
through vertex 70 and is parallel to base 50. The line 
should also be perpendicular to base line 74 of pyramid 
40. The camera's optical center should be positioned a 
distance X, or a distance substantially equal to X, from 
vertex 70 along line 72. Point 100 is located on base 50 
at a position where a line from vertex 70 perpendicularly 
intersects base 50. In a similar fashion, the optical cent- 



ers of cameras 56, 58 and 52 are positioned the distance 
X, or a distance substantially equal to X, along lines 76, 
80 and 84, respectively. 

[0018] FIG. 5 illustrates an eight-sided pyramid 120. 
5 Pyramid 1 20 has reflective surfaces 1 22 where each of 
surfaces 122 forms a 45 degree angle with a plane that 
passes through vertex 130 and is parallel to base 124. 
As with the four-sided pyramid of FIG. 2, each reflective 
surface of FIG. 5 may have a camera associated with 

10 it. Each camera's optical center is positioned on a line 
that is in a plane that passes through vertex 130 and is 
parallel to base 124. The line is perpendicular to base 
line 132 of the reflective surface associated with the 
camera to be positioned. Using an eight-sided pyramid 

is offers the advantage of using cameras with only a 45 
degree horizontal field of view to obtain a 360 degree 
view. Cameras with only a 45 degree field of view have 
inexpensive optics and enable a 360 degree view to be 
constructed using relatively inexpensive components. 

20 [0019] FIG. 6 is a top view of pyramid 120. As dis- 
cussed with regard to FIG. 5, each camera's optical 
center is positioned along a line 134 which is in a plane 
that passes through vertex 130 and is parallel to base 
124. The optical centers are positioned a distance X, or 

25 a distance substantially equal to X, along line 1 34 which 
is perpendicular to the aDpropriate base line 132. Point 
140 is on base 124 at the point of intersection between 
base 1 24 and a line that passes through vertex 1 30 and 
is perpendicular to base 124. 

30 [0020] Polyhedrons or pyramids having more or less 
reflective sides may be used. The advantage of using 
pyramids having a large number of sides is that cameras 
with moderate to small fields of view may be used. Cam- 
eras with moderate fields of view have relatively inex- 

35 pensive optics. The number of sides used in a pyramid 
is somewhat limited by the cost of providing a large 
number of cameras. A 360 degree view of a scene may 
be provided using a pyramid having three reflective 
sides. It may be expensive to use only a three-sided pyr- 

40 amid in order to provide a 360 degree field of view. This 
embodiment of the invention uses three cameras each 
with a 120 degree field of view, and cameras with such 
a wide field of view use relatively expensive optical com- 
ponents. 

^5 [0021] In applications where a full 360 degree view is 
not desired, it is possible to build a viewer that does not 
have a camera associated with each reflective surface 
of the pyramid. In addition to eliminating an unnecessary 
camera, it is also possible to eliminate an unnecessary 

50 pyramid polyhedron surface by using reflective ele- 
ments that are partial pyramids or partial polyhedrons. 
[0022] Although a pyramid configuration has been 
discussed in this example, different planar mirror ge- 
ometries may be used to redirect fields of view so that 

55 the cameras have virtual optical centers that are sub- 
stantially co-located. For example, solid, hollow or par- 
tial polyhedrons may be used. Additionally, in the case 
of a pyramid configuration the base and vertex do not 
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have to be physically present and can be thought of as 
conceptual aids such as a base plane or end and vertex 
point or end. 

[0023] FIG. 7 illustrates a block diagram of a system 
for controlling data produced by the cameras of a view- 
ing device such as the viewing device described in 
FIGS. 2 through 4. Cameras 52, 54, 56 and 58 obtain a 
360 degree view of an area via their associated reflec- 
tive surfaces of pyramid 40. The image signal or output 
signal of cameras 52, 54, 56 and 58 are passed through 
analog to digital converters (A/D) 160, 162, 164, and 
166, respectively. The output of the cameras can be 
thought of as a stream of pixels and the outnut of the A/ 
Ds can be thought of as data representative of the pixels 
from the cameras. The output of the A/Ds are passed 
through mux 170. Mux 170 allows the pixel data from 
each of the A/Ds to reach memory 172. Controller 174 
cycles the select lines of mux 170 so that the outputs of 
all of the A/Ds are stored in memory 172. Mux 170 is 
switched at a rate that is four times the pixel rate of the 
cameras. If more or less cameras are used, the rate at 
which mux 170 is switched will be increased or slowed 
accordingly, (t is also possible to eliminate mux 170 and 
to store the output of each A/D in a separate memory. 
Controller 174 is implemented using a microprocessor 
which provides control signals to counters that control 
the switching of mux 170 and counters used to provide 
addressing to memory 172. The control signals to the 
counters include reset, enable and a starting offset. 
[0024] As a result of the pixel information being 
passed to memory 172, memory 172 contains a 360 de- 
gree view of a scene. Pixel information stored in memory 
172 is passed through digital to analog converter (D/A) 
1 76 and to video display 1 78. The actual portion of mem- 
ory 172 that is passed to video display 178 via D/A 176 
is controlled via user input device 180. User input device 
180 may be a common device such as a mouse, joy- 
stick, or keyboard. The user may simply lean a joystick 
to the right to shift his view to the right, lean the joystick 
to the left to shift the view to the left, or leave the joystick 
in the center to keep the view unchanged. Based on the 
input from user device 180, controller 1 74 varies offsets 
and starting addresses that are used to provide ad- 
dressing to memory 172. 

[0025] FIG. 8 illustrates the relationship between the 
data provided by the cameras and the view available to 
the user. Since the cameras share a virtual optical cent- 
er, the view can be thought of as a cylindrical view. Sec- 
tor 200 can be thought of as representing the informa- 
tion provided by camera 52, sector 202 can be thought 
of as representing the information provided by camera 
54, sector 204 can be thought of as representing the 
information provided by camera 56, and sector 206 can 
be thought of as representing the information provided 
by camera 58. The surface of the cylinder in each sector 
can be thought of as a collection of columns, where each 
column is composed of pixels. For example, sector 200 
can be thought of as a collection of columns including 



columns 210, 212, 214, and 216. Likewise, the output 
produced by camera 54 can be thought of as a collection 
of columns which include column 218 in sector 202 and 
the output of camera 58 can include columns such as 

5 column 220 in sector 206. 

[0026] FIG. 9 illustrates how memory 172 is divided 
to provide easy access to different views based on sig- 
nals from user input device 1 80. Sections 230, 232, 234, 
and 236 correspond to sectors 206, 200, 202, and 204, 

10 respectively. Each of sections 230, 232, 234, and 236 
can be thought of as a block within memory 172. The 
blocks in memory 172 are broken into columns of se- 
quential addresses. The first column of memory seg- 
ment 230 corresponds to the first column of pixels of 

'5 sector 206. The number of memory positions associated 
with a column should be at least sufficient to have one 
location for each pixel in a particular column. For exam- 
ple, if a column of pixels from FIG. 8 includes 1000 pix- 
els, each column associated with the memory segments 

20 of FIG. 9 should have at least 1000 locations. The 
n umber of columns associated with a particular memory 
segment should be at least equal to the number of col- 
umns associated with a particular section of the cylinder 
of FIG. 8. 

25 [0027] If a camera scans in a horizontal direction, se- 
quential pixels are written in adjacent columns, but pos- 
sibly different rows, of a particular memory segment by 
simply changing an offset to a counter generated ad- 
dress. The overall write address is generated by adding 

30 the offset to the counter's output. This offset is changed 
at the rate in which the horizontally scanned pixels are 
received. After a horizontal scan is completed, the coun- 
ter is incremented and once again the offsets are 
changed at the horizontal scan rate. As a result, when 

35 addressing a particular segment of memory during a 
write cycle, the columns are addressed by changing the 
offset at the horizontal pixel scan rate, and incrementing 
the counter at the vertical scan rate. This type of ad- 
dressing scheme is used for accessing columns within 

40 each memory segment. When addressing different 
memory segments during a write cycle, a write segment 
offset is added to the sum of the counter output and the 
column offset. The write segment offset permits ad- 
dressing memory segments 230, 232, 234, and 236. 

45 The segment offset is changed at the same rate as mux 
170 is switched. 

[0028] Pixel data is read from memory 1 72 in a simi lar 
fashion. The sum of a counter output and two sets of 
offsets are used to generate a read address. Once an 

50 initial starting column has been picked, the read address 
is generated by switching a read column offset at a rate 
that is equal to the horizontal scan rate of a video dis- 
play. After reading one horizontal scans worth of data, 
the read counter is incremented and the read column 

55 offsets are changed at a rate equal to the horizontal scan 
rate of the display. As a result, the offset addresses are 
changing at the display's horizontal display rate and the 
counter is incremented at a rate equal to the vertical 
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scan rate of a display. It is possible to read data out at 
a rate faster or slower than required by the video display; 
however, if read out faster, a buffer memory should be 
used, if read out slower the video display may appear 
choppy to the viewer. 

[0029] It should be noted that the cylindrical arrange- 
ment of pixels of FtG. 8 is typically displayed on a flat or 
nearly flat display. As a result, the image may be dis- 
played by compensating for converting between a cylin- 
drical surface and a flat surface. This may be carried out 
using a simple conversion algorithm within a common 
digital signal processing integrated circuit. Methods for 
these types of conversions are well known in the art and 
can be found in "A Guided Tour of Computer Vision, 
Vishvjit S. Nalwa, Addison -Wesley Publishing Co., 
Reading, Massachusetts, 1993". It is also possible to 
carry out the conversion using a very high resolution dis- 
play. 

[0030] It should be noted that if the view selected by 
a user corresponds exactly to the view of a particular 
camera, such as camera 52, columns 240-248 are read 
from memory 1 70. Column 240 is the first column in seg- 
ment 232 and column 248 is the last column in segment 
232. If the user decides to move the view in a counter- 
clockwise direction, the start column will shift to the right 
so that the read operation begins at column 246 and 
ends at column 250. It should be noted that column 246 
is the second column associated with memory segment 
232 which has the pixel data from camera 52, and that 
column 250 is the first column of pixel data associated 
with camera 56. As the user shifts the view, the starting 
column shifts in relationship to the user's commands. 
For example, if the user indicates that the view should 
shift in a counter-clockwise direction, the start column 
of FIG. 9 moves to the right, similarly, if the viewer indi- 
cates that the view should shift in a clockwise direction, 
the start column shifts to the left. As before, columns are 
addressed by using offsets, if the offsets involve moving 
between memory segments, a read segment offset is 
added to the sum of the column offset and counter out- 
put. 

[0031] FIG. 10 illustrates a block diagram of controller 
174. Controller 174 includes microprocessor 270 and 
memory 272 Memory 272 includes RAM and ROM. 
Processor 270 receives commands on line 274 from us- 
er input device 180. Microprocessor 270 controls start, 
stop and reset of counter 276. Counter 276 controls the 
select lines of mux 170. Counter 276 counts at a rate 
that is four times the horizontal scan rate of the cameras. 
Write address generator 278 provides write addressing 
for memory 172. Write address generator 278 includes 
a counter, register for storing offsets and adder for add- 
ing the offsets and counter output. Microprocessor 270 
controls the offset selection and the counters used by 
write address generator 278. The write addresses are 
formed as described with regard to FIG. 9. Read ad- 
dress generator 280 provides read addresses to mem- 
ory 172. Read address generator 280 includes a coun- 



ter, register for storing offsets and adder for adding the 
offsets and counter output. As with write address gen- 
erator 278, microprocessor 270 controls the offset se- 
lection and the counters of read address generator 280. 
5 Microprocessor 270 also controls the starting column 
used by the counters based on inputs provided on line 
274 from user input 180. 

[0032] The write and read addresses are provided to 
memory 172 separately if memory 172 is implemented 
10 using a two port memory. If memory 1 72 is implemented 
with a single port memory, the write and read addresses 
are multiplexed to memory 172. 

[0033] FIG. 11 illustrates the viewing system of FIG. 
2 with a fifth camera. Camera or image gathering device 
75 400 is located in pyramid 40 with the optical center of 
camera 400 located at, or nearly at, virtual optical center 
90. Camera 400 views objects in the direction of arrow 
410. The resulting view coupled with the views of the 
remaining four cameras, provides a nearly spherical 
view, ff the cameras of FIG. 11 are replaced with image 
producing devices, the nearly spherical viewing system 
becomes a nearly spherical projection system. It should 
be noted, that a camera or projection device, may be 
placed at the virtual optical center of viewing/projection 
devices having pyramids with three, four or more sides. 
It should also be noted that base edges 420 of the re- 
flective surfaces should be beveled to avoid undesirable 
obstruction of camera 400's field of view. It is also pos- 
sible to avoid undesirable image artifacts from base 
edges 420 by moving camera or image processing de- 
vice 400. Device 400 should be moved so that device 
400's optical center is positioned away from virtual op- 
tical center 90 in the direction of arrow 41 0. Device 400's 
optical center should be positioned so that the device's 
used field of view does not include edges 420. 
[0034] FIG. 1 2 illustrates a top view of the pyramid of 
FIG. 2. In reference to FIG. 2, camera 52, 54, 56, and 
58 have been moved upward in the direction of base 50. 
As a result, virtual optical centers 500, 502, 504 and 506, 
which correspond to cameras 52, 54, 56 and 58, respec- 
tively, are moved away from virtual optical center 90. It 
is desirable to move the virtual optical centers so that 
camera 52 captures an image between lines 508 and 
5 1 0 that are unaffected by an edge of the pyramid, cam- 
era 54 captures an image between lines 512 and 514 
that are unaffected by an edge of the pyramid, camera 
56 captures an image between lines 516 and 518 that 
are unaffected by an edge of the pyramid, and camera 
58 captures an image between lines 520 and 522 that 
are unaffected by an edge of the pyramid. This results 
in the cameras not capturing images distorted by edges 
of the pyramid from narrow planar shaped regions. In 
particular, planar regions 524. 526, 528, and 530 are not 
used and form blind regions. This offers the advantage 
of removing image regions that are distorted by the edg- 
es of the reflective pyramid. Eliminating these portions 
of the fields of view alleviates the need to provide image 
processing that compensates for image artifacts at the 
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edges. It is desirable to keep virtual optical centers 500 t 
502, 504, and 506 closely clustered so that planes 524, 
526, 528, and 530 are only as thin as necessary to avoid 
edge artifacts. By maintaining such thin planes, the 
need to process the images at their common boundaries 
is removed while minimizing the noticeable effect seen 
by a user. 

[0035] FIG. 13 illustrates the pyramid of FIG 12 with 
shades 560, 562, 564, and 566 positioned in planar re- 
gions 524, 526, 528, and 530, respectively. The shades 
reduce the amount of unwanted light that enters the 
cameras. Similar shades may be placed in blind regions 
between device 400 and one or more of the other image 
processing devices. It is also possible to place a shade 
on base 50 with the edges of the shade extending be- 
yond the edges of the base to reduce the amount of un- 
wanted light that enters cameras 52, 54, 56, and 58 from 
sources behind base 50. 

[0036] FIG. 14 illustrates reflective pyramids 602 and 
604 arranged in a base-to-base configuration. The bas- 
es may be in contact with each other or spaced apart. 
Reflected pyramids 602 and 604 each have four reflec- 
tive side facets. Pyramid 602 has reflective side facets 
608, 610, 612, and 614. Reflective pyramid 604 has re- 
flective sides 616, 618, 620, and 622. Pyramid 602 in- 
cludes vertex 624 and pyramid 604 includes vertex 626. 
Vertices 624 and 626 are on a line 628 that is perpen- 
dicular to the base of each pyramid. Each pyramid has 
four image processing devices such as cameras with a 
field of view being redirected by a reflective surface. 
With regard to pyramid 602, a camera with an optical 
center positioned at point 630 has a field of view in the 
direction of arrow 632 where that field of view is redi- 
rected by reflective surface 608. A second camera with 
an optical center at point 634 has a field of view in the 
direction of arrow 636 which is redirected by reflective 
surface 610. A third camera with an optical center at 
point 638 has a field of view in the direction of arrow 640 
which is redirected by reflective surface 612. A fourth 
camera with an optical center at point 642 has a field of 
view in the direction of arrow 644 which is redirected by 
reflective surface 614. Regarding reflective pyramid 
604, a first camera with an optical center at point 646 
has a field of view in the direction of arrow 648 which is 
redirected by reflective surface 616. A second camera 
with an optical center at point 650 has a field of view in 
the direction of arrow 652 which is redirected by surface 
618. A third camera with an optical center at point 654 
has a field of view in the direction of arrow 656 which is 
redirected by reflective surface 620. A fourth camera 
with an opticat center at point 658 has a field of view in 
the direction of arrow 660 which is redirected by reflec- 
tive surface 622. The cameras associated with each of 
the pyramids are positioned in a way similar to how the 
cameras were positioned with regard to FIGS. 2, 3, 4, 
1 1 , and 1 2 so that each set of four cameras shares a 
common virtual optical center or have closely clustered 
virtual optical centers within their associated pyramid. 



Each set of cameras may also have offset virtual optical 
centers within their associated pyramid. The cameras 
may be positioned so that the cameras associated with 
each pyramid share a common virtual optical center 
5 along line 628 where the bases of the two pyramids 
meet. It is also possible to position the cameras so that 
their offset virtual optical center are clustered about a 
point on line 628 where the bases of the two pyramids 
meet. 

w [0037] The structure of FIG. 14 increases the vertical 
field of view as compared to the viewers discussed with 
regard to FIGS. 2, 3, and 4. The viewer of FIG. 14 in- 
creases the vertical field of view by using two cameras 
rather than one camera for the same or nearly the same 
'5 vertical dimension. It should be noted that a projector 
may be constructed by replacing the cameras with im- 
age producing devices. It should also be noted that re- 
flective pyramids 602 and 604 may be rotationally mis- 
aligned with respect to each other. This misaligned re- 
lationship is obtained by rotating one or both of the pyr- 
amids about an axis that passes through the vertices of 
both pyramids. For example, the axis may be co-linear 
with line 628. As a result of this rotation, the side edges 
of the reflective side facets of pyramid 602 will not align 
with the side edges of the reflective side facets of pyra- 
mid 604. 

[0038] Although a pyramid configuration has been 
discussed in this example, different planar mirror ge- 
ometries may be used to redirect fields of view so that 
the cameras have virtual optical centers that are sub- 
stantially co-located. For example, solid, hollow or par- 
tial polyhedrons may be used. Additionally, in the case 
of a pyramid configuration the base and vertex do not 
have to be physically present and can be thought of as 
conceptual aids such as a base plane or end and vertex 
point or end. 

[0039] FIG. 1 5 illustrates two reflective pyramids. Re- 
flective pyramid 702 is nested within reflective pyramid 
704. It should be noted that more than two reflective pyr- 
amids may be nested. For example, another reflective 
pyramid may be nested within reflective pyramid 702 
and yet another reflective pyramid may be nested within 
the pyramid that is nested within pyramid 702. Vertex 
706 of pyramid 702 and vertex 708 of pyramid 704 are 
on a line 710 which is perpendicular to the bases of both 
pyramids. Once again, each pyramid includes four im- 
age processing devices such as cameras each with a 
field of view that is redirected by a reflective surface of 
their associated pyramid. Pyramid 702 includes reflec- 
tive side facets 712, 714, 716, and 718. Reflective pyr- 
amid 704 includes reflective side facets 720, 722, 724, 
and 726. Four cameras are positioned so that their field 
of view is redirected by the reflective surfaces of pyramid 
702. A first camera with an optical center at point 730 
and a field of view in direction of arrow 732 has its field 
of view redirected by reflective surface 712. A second 
camera with an optical center at point 734 and a field of 
view in the direction of arrow 736 has its field of view 
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redirected by reflective surface 714. A third camera with 
an optical center at point 738 and a field of view in the 
direction of arrow 740 has its field of view redirected by 
reflective surface 716. A fourth camera with an optical 
center at point 742 and a field of view in the direction of 
arrow 744 has its field of view redirected by reflective 
surface 718. It should be noted that pyramid 702 and its 
associated cameras are positioned so that the field of 
view of the cameras is not obstructed by pyramid 704. 
This is accomplished by allowing pyramid 702 to extend 
beyond the base of pyramid 704. Regarding pyramid 
704, a first camera with an optical center at point 750 
and a field of view in the direction of arrow 752 has its 
field of view redirected by reflective surface 720. A sec- 
ond camera with an optical center at point 754 and a 
field of view in the direction of arrow 756 has its field of 
view redirected by reflective surface 722. A third camera 
with an optical center at point 758 and a field of view in 
the direction of arrow 760 has its field of view redirected 
by reflective surface 724. A fourth camera with an optical 
center at point 762 and a field of view in the direction of 
arrow 764 has its field of view redirected by reflective 
surface 726. The cameras associated with each of the 
pyramids are positioned in accordance with the position- 
ing illustrated with FIGS. 2, 3, 4, 11, and 12 so that the 
eight cameras share a virtual optical center at position 
770 or have closely clustered virtual optical centers with- 
in pyramid 702. Each set of cameras may also have off- 
set virtual optical centers within pyramid 702. 
[0040] The panoramic viewer of FIG. 15 can be pro- 
vided with a ninth camera having an optical center at 
point 770 and a field of view in the direction of arrow 772 
to provide a viewer with a partially-spherical view. The 
camera having an optical center at position 770 may use 
a wide-angle lens to provide a broader view. 
[0041] FIG. 16 illustrates the partially-spherical view- 
er of FIG. 1 5 with an additional camera having an optical 
center at point 780 and a field of view in the direction of 
arrow 782 where that field of view is redirected by planar 
mirror 784. It should be noted that optical center 780 is 
on line 710 which passes through the vertices of pyra- 
mid 702 and 704 as well as virtual optical center 770. It 
should also be noted that point 780 is placed a distance 
away from planar mirror 784 that is equal or nearly equal 
to the distance between planar mirror 784 and virtual 
optical center 770. By placing a camera with an optical 
center at point 780 and having the field of view redirect- 
ed by planar mirror 784, the partialty-spherical viewer of 
FIG. 15 becomes a spherical viewer. In order to increase 
the field of view of the camera positioned with an optical 
center at point 780, the camera may be provided with a 
wide-angle lens. It should also be noted that planar mir- 
ror 784 may be replaced with a curved mirror to provide 
a wider field of view for the camera positioned at point 
780 and minimize the need for a wide-angle lens. 
[0042] Although a pyramid configuration has been 
discussed in this example, different planar mirror ge- 
ometries may be used to redirect fields of view so that 



the cameras have virtual optical centers that are sub- 
stantially co-located. For example, solid, hollow or par- 
tial polyhedrons may be used. Additionally, in the case 
of a pyramid configuration the base and vertex do not 
5 have to be physically present and can be thought of as 
conceptual aids such as a base plane or end and vertex 
point or end. 

[0043] Regarding FIGS. 1 5 and 1 6, it should be noted 
that a projector may be constructed by replacing the 

10 cameras with image producing devices. It should also 
be noted that reflective pyramids 702 and 704 may be 
rotationally misaligned with respect to each other. This 
misaligned relationship is obtained by rotating one or 
both of the pyramids about an axis that passes through 

'5 the vertices of both pyramids. For example, the axis may 
be co-linear with line 71 0. As a result of this rotation, the 
side edges of the reflective side facets of pyramid 702 
will not align with the side edges of the reflective side 
facets of pyramid 704. 

20 [0044] FIG. 17 illustrates a stand used to support a 
panoramic viewer. Reflective pyramid 800 is mounted 
to stand or post 802 using a support member such as 
hollow tube 804. The pyramid is secured to hoilow tube 
804 at vertex end 806. The hollow tube is secured to 

25 stand 802 by angle brackets 808. Hollow tube 804 ex- 
tends beyond vertex end 806 so that cameras 810 may 
be supported by tube 804. The cameras are mounted 
to tube 804 by strap or belt 81 2 which presses cameras 
810 against spacer 814. The pressure provided by 

30 clamp or strap 812 provides friction between camera 
810, spacer 814, and the outer surface of tube 804 and 
thereby mounts cameras 810 to tube 804 in a secure 
fashion. It is also possible to provide a second strap and 
associated spacers at end-section 81 6 of cameras 81 0. 

35 Video and power connections to cameras 810 are pro- 
vided by cables 818 which are fed through hollow tube 
804 and out through space 820 which is between post 
802 and the base of pyramid 800. It should be noted that 
hollow tube 804 may be replaced with a solid support 

40 member; however, a hollow support member provides 
a convenient path for routing cables. Feeding the cables 
through tube 804 prevents the cables from entering the 
field of view of cameras 810. Rubber stands or feet 824 
are provided at the base end of pyramid 800. These 

45 stands may be used in place of post 802 to provide flex- 
ibility in application where the user does not want to use 
post 802. 

[0045] It is also possible to invert the viewer of FIG. 
17 so that the viewer is supported by end 830 of tube 

so 804. In this configuration cables 818 will simply be 
passed out through an opening at end 830 of tube 804. 
In this configuration tube 804 is mounted to post 802 at 
end 830 using angle brackets similar to angle brackets 
808. It is also possible to mount end 830 to any conven- 

55 ient structure to support the panoramic viewer. 

[0046] The stand of FIG. 1 7 is applicable to the viewer 
of FIGS. 14, 15, and 16. As discussed with regard to 
FIG. 17, the viewer is mounted to a hollow tube passing 
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through the vertices or vertex ends of both pyramids. 
Calibration 

[0047] A higher quality image may be produced by s 
calibrating the camera system. Calibration may be used 
to determine image mapping functions (which may be 
implemented as look up tables) that compensate for dif- 
ferent types of image distortion. For example, the map- 
ping functions may be used to correct barrel distortion 10 
which is a distortion introduced by a wide-angle lens. 
Mapping functions may also be used to correct other 
types of distortions such as a rotational distortion result- 
ing from misaligned charged coupled devices within the 
cameras. FIG. 18 illustrates a combination of barrel dis- is 
tortion and rotational distortion, where the distortion re- 
sults in rectangular object 900 appearing as distorted 
image 902. Distorted image 902 is rotated with respect 
to undistorted object 900 and a barrel distortion is seen 
where edges 904 and 906 of rectangular object 900 ap- 20 
pear as edges 908 and 910 of image 902, and where 
edges 912 and 914 appear as edges 916 and 918, re- 
spectively. This distortion may be corrected using map- 
ping functions that are determined by calibrating the 
camera system. 25 
[0048] FIG. 19 illustrates a process for calibrating the 
camera system. A vertical column of equally spaced el- 
liptical dots 930 is placed in a fixed position. The dots 
are white on a black background where the major axis 
of the elliptical dot is in the vertical direction. Panoramic 30 
camera 940 is then rotated in small discrete steps about 
an axis 942 passing through the virtual optical center of 
the panoramic camera. At each step, the distorted im- 
age of the column of elliptical dots is viewed and a map- 
ping function for that data is determined to remove the 35 
distortion. This function maps the image such that the 
distorted image of each vertical column of equally 
spaced dots is a vertical column of equally spaced dots 
in the mapped image. Note that although the images of 
the white dots have black gaps between them, the map- 40 
ping function is computed to apply to every image pixel 
(including the pixels between the white dots) through in- 
terpolation. An image mapping function is determined 
at each of the discrete steps as the camera is rotated, 
the union of these mapping functions is combinable into 45 
a 2-D to 2-D mapping that ensures not only that each 
vertical column of equally spaced dots appears as a ver- 
tical column of equally spaced dots in the image, but 
also that these columns are spaced horizontally in pro- 
portion to the angular rotation between their image ac- so 
quisitions, the latter providing a cylindrical (rather than 
flat) image of the scene. 

[0049] FIGS. 20 and 21 illustrate how the data repre- 
sentative of the vertical column of elliptical dots is rep- 
resented as a result of distortion. FIG. 20 is similar to 55 
FIG. 8 in that it illustrates the relationship between the 
data provided by the cameras and the view available to 
the user. It should be noted that the vertical column of 



dots is not in a single column of FIG. 20 as a result of 
the distortion. The distortion has caused the dots to oc- 
cupy columns 960, 962, 964 and 966, rather than just a 
single column. FIG. 21 is similar to FIG. 9 in that it illus- 
trates how image data is stored. When the distorted im- 
age data is stored in memory as represented in FIG. 21 , 
the data representative of the dots also occupies several 
columns where columns 980, 982, 984 and 986 corre- 
spond to the columns 960, 962, 964 and 966 of FIG. 20, 
respectively. The image mapping function determined 
during the calibration phase is used to correct for this 
distortion when the data is read from the memory rep- 
resented in FIG. 21 . The corrected or undistorted image 
data may then be displayed to the user or written to a 
memory used to store data representative of the undis- 
torted image. FIG. 22 illustrates the relationship be- 
tween the data read from the memory of FIG. 21 and 
the undistorted view made available to a user. For ex- 
ample, the mapping function associated with column 
1000 specifies that when reading data for use in the up- 
permost portion of column 1000, data is read from col- 
umn 980 and when reading data for use in the portion 
of column 1000 just below, data is read from column 
982. The mapping function also specifies that when 
reading data for use in the middle portion of column 
1000, data is read from column 984. Moving further 
down column 1000, data is then read from column 982, 
then column 980, and eventually from column 986 when 
data for use at the bottom of column 1000 is retrieved. 
As a result of reading data, as specified by the mapping 
function, the column of data will appear vertical to a user 
viewing a display. FIG. 22 illustrates that the data re- 
trieved from the memory of FIG. 21 now appears as a 
vertical column where the distortion is no longer evident. 
A similar mapping function, as determined during cali- 
bration, is used for each column of FIG 22 to produce 
an undistorted image for display. It should be noted that 
multiple discrete rotational steps used to calibrate the 
panoramic camera could be substituted by a group of 
several columns illustrated in FIG. 22. 
[0050] Color and intensity calibration may also be car- 
ried out using a procedure similar to the procedure illus- 
trated in FIG. 19. In this case, column 930 of elliptical 
dots is replaced by a known color pattern. The pano- 
ramic camera is then rotated so that each camera cap- 
tures an image of the color pattern. Several color pat- 
terns (such as various shades of red, green, blue, and 
gray) could be used one by one. Then on a pixel-by- 
pixel basis, the data from each camera is adjusted to 
correct any red, green, or blue distortion so that the pro- 
duced image has a color pattern that closely matches 
the calibration color pattern. Additionally, the intensity of 
each pixel from each camera is adjusted so that there 
is relatively uniform intensity and color within a single 
camera's image and between the images of the multiple 
cameras when viewing a scene with constant color and 
brightness. As discussed with regard to the mapping 
function, the pixel-by-pixel adjustment may be stored in 
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a table. A less precise, but simpler method of color and 
intensity calculation may be used. This method simply 
involves manually adjusting the color and intensity con- 
trols of each camera to get correct color and intensity 
when viewing a scene with a particular color and inten- 
sity. It should be noted that by using this method, all of 
the pixels of a particular camera receive the same ad- 
justments. 

[0051] FIG. 23 illustrates a panoramic camera system 
where calibration based image mapping correction is 
used. FIG. 23 is similar to FIG. 7; however, it should be 
noted that a frame buffer memory and an additional mi- 
croprocessor have been included. Cameras 52, 54, 56, 
and 58 gather image data and then pass the data to an- 
alog-to-digital converters 160, 162, 164, and 166, re- 
spectively. The output of the analog-to-digital converters 
are then passed through red, green, blue, and intensity 
adjustment units 1010, 1012, 1014, and 1018. It is pos- 
sible to place these units before the analog/digital con- 
verters, if the adjustment units are analog units. Addi- 
tionally, it is also possible to use cameras that have the 
adjustment units built into each camera. In any case, the 
adjustment units are programmed or set to adjust the 
color and intensity as determined by the calculation pro- 
cedures. Each of these units adjust the red, green, and 
blue levels and the overall levels of the signals from the 
analog-to-digital converter. It should be noted that if 
cameras 52 through 58 are color cameras, analog-to- 
digital converters 160 to 166 typically receive three sig- 
nals and output three signals, where each pair of input 
and output signals corresponds to one of the colors red, 
green, and blue. Units 1010 through 1016 simply adjust 
the relative amplitudes of the red, green, and blue sig- 
nals in accordance with the settings determined during 
the calibration procedure. Each of units 1010 through 
1018 also adjust the overall amplitude of the red, green, 
and blue signals in accordance with the overall intensity 
calibration settings. The outputs of the red, green, and 
blue intensity adjustments are then passed through a 
multiplexer as discussed in FIG. 7, and are passed to 
frame buffer memory 1 030. It is also possible to replace 
frame buffer 1030 with an individual frame buffer for 
each of red, green, blue and intensity units 1010, 1012, 
1014, and 1018. The outputs of each of the individual 
frame buffer may then be passed to microprocessor 
1030 via multiplexer 170. 

[0052] Frame buffer memory 1030 is operated in a 
fashion similar to memory 172 of FIG. 7 and stores the 
data representing the distorted images as was dis- 
cussed in reference to FIG. 21. Microprocessor 1040 
then reads the data from frame buffer memory 1 030 us- 
ing the mapping functions determined during the cali- 
bration procedure and then writes the data into display 
memory 1050. Recalling the discussion associated with 
FIG. 22, the data representing undistorted images is 
then stored in memory 1050 for retrieval by the users. 
The users can retrieve the data as is discussed in ref- 
erence to FIG. 7 where the data read out is determined 
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based on a user's input. It is also possible for the entire 
contents of display memory to be made available to 
each user. The data may be communicated to each user 
through a communication network such as a telephone 
network or a data network, or it may be directly commu- 
nicated to the user via a dedicated wired or wireless 
communication path. The user then may use a digital- 
to-analog converter to convert the data into an analog 
format that may be displayed for the user or the user 
may use the digital data directly and forego the use of 
a digital-to-analog converter. 



Claims 

1. A panoramic viewing apparatus, CHARACTER- 
IZED BY: 

a plurality of first image processing devices, 
each having an optical center and a field of 
view; 

a plurality of second image processing devices, 
each having an optical center and a field of 
view; and 

a first and a second reflective element each be- 
ing at least partially polyhedral arranged adja- 
cent to each other, the first reflective element 
having a plurality of first reflective facets facing 
in different directions, each of at least two of the 
plurality of first reflective facets redirecting a 
field of view of one of the plurality of first image 
processing devices to create a plurality of first 
virtual optical centers, and the second reflec- 
tive element having a plurality of second reflec- 
tive facets facing in different directions, each of 
at least two of the plurality of second reflective 
facets redirecting a field of view of one of the 
plurality of second image processing devices to 
create a plurality of second virtual optical cent- 
ers. 

2. A panoramic viewing apparatus, CHARACTER- 
IZED BY: 

a plurality of first image processing devices, 
each having an optical center and a field of 
view; 

a plurality of second image processing devices, 
each having an optical center and a field of 
view; 

a first reflective element being at least partially 
polyhedral: and 

a second reflective element being at least par- 
tially polyhedral with at least a portion of the 
second reflective element being positioned 
within the first reflective element, the first reflec- 
tive element having a plurality of first reflective 
facets facing in different directions, each of at 
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least two of the plurality of first reflective facets 
redirecting a field of view of one of the plurality 
of first image processing devices to create a 
plurality of first virtual optical centers, and the 
second reflective element having a plurality of s 
second reflective facets facing in different di- 
rections, each of at least two of the plurality of 
second reflective facets redirecting a field of 
view of one of the plurality of second image 
processing devices to create a plurality of sec- io 
ond virtual optical centers. 

The panoramic viewing apparatus of claim 1 or 2, 
characterized in that the plurality of either first virtual 
optical centers, or second virtual optical centers, or is 
first and second virtual optical centers are substan- 
tially co-located. 

The panoramic viewing apparatus of claim 1 or 2, 
characterized in that at least two of the plurality of so 
first virtual optical centers are offset from each other 
to create a first blind region that encompasses at 
least one edge of a first reflective facet that redirects 
a field of view of a first image processing device as- 
sociated with a first virtual optical center, and at 25 
least two of the plurality of second virtual optical 
centers are offset from each other to create a sec- 
ond blind region that encompasses at least one 
edge of a second reflective facet that redirects a 
field of view of a second image processing device 30 
associated with a second virtual optical center. 

The panoramic viewing apparatus of claim 1 or 2, 
characterized in that the first image processing de- 
vices are either image capture devices, or image 35 
producing devices, and/or that the second image 
processing devices are either image capture devic- 
es, or image producing devices. 

The panoramic viewing apparatus of claim 2, further 40 
characterized by a third image processing device 
that is either positioned within the second reflective 
element, or has a field of view in the direction of the 
first reflective element; and 

a reflective surface facing the third image 45 
processing device and positioned between the third 
image processing device and the first reflective el- 
ement. 

The panoramic viewing apparatus of claim 6, further 50 
characterized by a fourth image processing device 
positioned within the second reflective element. 

The panoramic viewing apparatus of claim 6, char- 
acterized in that the reflective surface is curved. 55 
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